Dynamic Mapping Method Based Speech Driven Face Animation System
نویسندگان
چکیده
In the paper, we design and develop a speech driven face animation system based on the dynamic mapping method. The face animation is synthesized by the unit concatenating, and synchronous with the real speech. The units are selected according to the cost functions which correspond to voice spectrum distance between training and target units. Visual distance between two adjacent training units is also used to get better mapping results. Finally, the Viterbi method is used to find out the best face animation sequence. The experimental results show that synthesized lip movement has a good and natural quality.
منابع مشابه
Expressive Face Animation Synthesis Based on Dynamic Mapping Method
In the paper, we present a framework of speech driven face animation system with expressions. It systematically addresses audio-visual data acquisition, expressive trajectory analysis and audio-visual mapping. Based on this framework, we learn the correlation between neutral facial deformation and expressive facial deformation with Gaussian Mixture Model (GMM). A hierarchical structure is propo...
متن کاملReal-time speech-driven face animation with expressions using neural networks
A real-time speech-driven synthetic talking face provides an effective multimodal communication interface in distributed collaboration environments. Nonverbal gestures such as facial expressions are important to human communication and should be considered by speech-driven face animation systems. In this paper, we present a framework that systematically addresses facial deformation modeling, au...
متن کاملIndividual 3D Face Synthesis Based on Orthogonal Photos and Speech-Driven Facial Animation
In the paper, a methodology for individual face synthesis using given orthogonal photos is proposed. And an integrated speech-driven facial animation system is presented. Firstly, in order to capture given subject’s personal facial configuration, a novel coarse-to-fine strategy based on facial texture and deformable template is proposed to localize some facial feature points in the image of fro...
متن کاملA Speech Driven Face Animation System Based on Machine Learning
Lip synchronization is the key issue in speech driven face animation system. In this paper, some clustering and machine learning methods are combined together to estimate face animation parameters from audio sequences and then apply the learning results to MPEG-4 based speech driven face animation system. Based on a large recorded audio-visual database, an unsupervised cluster algorithm is prop...
متن کاملAn Integrated Framework for Face Modeling , Facial Motion
This dissertation presents an integrated framework for face modeling, facial motion analysis, and facial motion synthesis. This framework systematically addresses three closely related research issues: (1) selecting a quantitative representation of facial deformation for face modeling and animation; (2) automatic facial motion analysis based on the same visual representation; and (3) speech-to-...
متن کامل